Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(currency_conversion): re frame the currency_conversion crate to make api calls on background thread #6906

Open
wants to merge 18 commits into
base: main
Choose a base branch
from

Conversation

prajjwalkumar17
Copy link
Contributor

@prajjwalkumar17 prajjwalkumar17 commented Dec 20, 2024

Type of Change

  • Bugfix
  • New feature
  • Enhancement
  • Refactoring
  • Dependency updates
  • Documentation
  • CI/CD

Description

Due to redis lock being held for a long time, which made the forex api-calls synchronous, we had to re-frame the way how things were being operated for forex rates fetch, The changes includes:

  1. Acquiring redis lock, just before api-call and releasing it once data is written on redis.
  2. Always responding back with stale-data and operating everything (redis-write, update, api call) on background thread.
  3. Throwing error, if api_keys are missing in configs, whenever call is being made.
  4. Greater log coverage.

Additional Changes

  • This PR modifies the API contract
  • This PR modifies the database schema
  • This PR modifies application configuration/environment variables

Motivation and Context

How did you test it?

  1. Testing for first call when there is no data present in cache as well as redis:
    Screenshot 2025-01-03 at 4 35 53 PM
curl --location 'http://127.0.0.1:8080/forex/rates' \
--header 'api-key: xxxxxxx' \
--data ''
Screenshot 2025-01-03 at 5 13 35 PM
  1. Testing for the case, when data is present in cache and/or redis:
Screenshot 2025-01-03 at 5 14 59 PM
  1. For cases when data is expired in cache, it will provide us with the stale data and a background thread will be spun which will update the data on cache as well as redis.

Full response:

{
    "data": {
        "base_currency": "USD",
        "conversion": {
            "TWD": {
                "to_factor": "32.939501",
                "from_factor": "0.0303586869758591667797274767"
            },
           ......
            "MOP": {
                "to_factor": "8.011576",
                "from_factor": "0.1248193863479545098242842607"
            }
        }
    },
    "timestamp": 1735904610
}

Checklist

  • I formatted the code cargo +nightly fmt --all
  • I addressed lints thrown by cargo clippy
  • I reviewed the submitted code
  • I added unit tests for my changes where possible

@prajjwalkumar17 prajjwalkumar17 self-assigned this Dec 20, 2024
@prajjwalkumar17 prajjwalkumar17 requested a review from a team as a code owner December 20, 2024 06:29
Copy link

semanticdiff-com bot commented Dec 20, 2024

Review changes with  SemanticDiff

Changed Files
File Status
  crates/router/src/core/currency.rs  85% smaller
  crates/router/src/configs/settings.rs  28% smaller
  crates/router/src/utils/currency.rs  27% smaller
  config/config.example.toml Unsupported file format
  config/deployments/env_specific.toml Unsupported file format
  config/development.toml Unsupported file format
  config/docker_compose.toml Unsupported file format
  crates/analytics/docs/README.md Unsupported file format
  loadtest/config/development.toml Unsupported file format

@prajjwalkumar17 prajjwalkumar17 marked this pull request as draft December 20, 2024 06:30
@prajjwalkumar17 prajjwalkumar17 marked this pull request as ready for review January 3, 2025 11:59
@prajjwalkumar17 prajjwalkumar17 requested a review from a team as a code owner January 3, 2025 11:59
config/config.example.toml Outdated Show resolved Hide resolved
crates/router/src/configs/settings.rs Show resolved Hide resolved
crates/router/src/configs/settings.rs Outdated Show resolved Hide resolved
crates/router/src/utils/currency.rs Outdated Show resolved Hide resolved
crates/router/src/utils/currency.rs Outdated Show resolved Hide resolved
crates/router/src/utils/currency.rs Outdated Show resolved Hide resolved
crates/router/src/utils/currency.rs Outdated Show resolved Hide resolved
crates/router/src/utils/currency.rs Outdated Show resolved Hide resolved
crates/router/src/utils/currency.rs Outdated Show resolved Hide resolved
@prajjwalkumar17 prajjwalkumar17 requested a review from a team as a code owner January 6, 2025 10:31
Chethan-rao
Chethan-rao previously approved these changes Jan 9, 2025
@prajjwalkumar17 prajjwalkumar17 changed the title refactor(currency_conversion): re frame the crate to make api calls on background thread refactor(currency_conversion): re frame the currency_conversion crate to make api calls on background thread Jan 10, 2025
crates/analytics/docs/README.md Outdated Show resolved Hide resolved
crates/router/src/utils/currency.rs Outdated Show resolved Hide resolved
@Gnanasundari24 Gnanasundari24 added this pull request to the merge queue Jan 22, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to no response for status checks Jan 22, 2025
@Gnanasundari24 Gnanasundari24 added this pull request to the merge queue Jan 22, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jan 22, 2025
Copy link

codecov bot commented Jan 24, 2025

Codecov Report

Attention: Patch coverage is 0% with 94 lines in your changes missing coverage. Please review.

Please upload report for BASE (main@7fd3551). Learn more about missing BASE report.

Files with missing lines Patch % Lines
crates/router/src/utils/currency.rs 0.00% 86 Missing ⚠️
crates/router/src/core/currency.rs 0.00% 8 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #6906   +/-   ##
=======================================
  Coverage        ?    4.12%           
=======================================
  Files           ?     1010           
  Lines           ?   207377           
  Branches        ?        0           
=======================================
  Hits            ?     8548           
  Misses          ?   198829           
  Partials        ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@likhinbopanna likhinbopanna added this pull request to the merge queue Jan 24, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to no response for status checks Jan 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-currency-conversion Area: Currency Conversion C-refactor Category: Refactor
Projects
None yet
Development

Successfully merging this pull request may close these issues.

refactor(currency_conversion): re-frame the crate to make api calls on background thread
4 participants